<html>
<head><meta charset="utf-8"><title>Tests for command line parsing · t-compiler/help · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/index.html">t-compiler/help</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html">Tests for command line parsing</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="243933698"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/243933698" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eholk <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#243933698">(Jun 25 2021 at 15:46)</a>:</h4>
<p>Hi all, apologies if this is not the right place for this question. I'm working on a small fix around parsing arguments like <code>-A clippy:foo</code>, and was wondering where the best place to put tests for that is.</p>



<a name="243933903"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/243933903" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eholk <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#243933903">(Jun 25 2021 at 15:48)</a>:</h4>
<p>My first thought is to stick them in a <code>#[cfg(test)]</code> module inside the file I changed, but I saw there's the ui test suite also, which seems like it might be a better place for that.</p>



<a name="243934539"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/243934539" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> fee1-dead <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#243934539">(Jun 25 2021 at 15:52)</a>:</h4>
<p>Would you mind sharing the specific issue that you are working on?</p>



<a name="243935204"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/243935204" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eholk <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#243935204">(Jun 25 2021 at 15:57)</a>:</h4>
<p>I don't think there's an issue yet, but it came up when discussing <a href="https://github.com/rust-lang/cargo/issues/5034#issuecomment-867838169">https://github.com/rust-lang/cargo/issues/5034#issuecomment-867838169</a>. Basically, rustc doesn't ignore <code>clippy::</code> on the command line when clippy is not enabled like it does when lints are specified through attributes.</p>



<a name="243935269"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/243935269" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eholk <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#243935269">(Jun 25 2021 at 15:58)</a>:</h4>
<p>I should probably file an issue about this specific problem.</p>



<a name="243937072"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/243937072" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eholk <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#243937072">(Jun 25 2021 at 16:10)</a>:</h4>
<p>Okay, it has its own issue now: <a href="https://github.com/rust-lang/rust/issues/86628">https://github.com/rust-lang/rust/issues/86628</a></p>



<a name="243946134"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/243946134" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#243946134">(Jun 25 2021 at 17:29)</a>:</h4>
<p>A <code>ui</code> test is usually the way to go. That will let you control the flags passed to <code>rustc</code> and validate the expected output (either an error message or a successful build). This part of the rustc dev guide might be useful to you: <a href="https://rustc-dev-guide.rust-lang.org/tests/adding.html#header-commands-configuring-rustc">https://rustc-dev-guide.rust-lang.org/tests/adding.html#header-commands-configuring-rustc</a></p>
<p>In particular, you will probably need the <code>// compile-flags: </code> and <code>// build-pass</code> directives.</p>



<a name="243946780"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/243946780" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eholk <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#243946780">(Jun 25 2021 at 17:34)</a>:</h4>
<p>Oh, perfect. That looks like exactly what I want.</p>



<a name="245106396"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/245106396" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eholk <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#245106396">(Jul 06 2021 at 22:27)</a>:</h4>
<p>So resurrecting this thread, I added a test that's supposed to fail and an stderr file for the expected output. However, the ui tests are still failing, but this time with the errors in JSON format. Is there something special that needs to be done for JSON output?</p>



<a name="245199786"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/245199786" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#245199786">(Jul 07 2021 at 15:36)</a>:</h4>
<p><span class="user-mention" data-user-id="421986">@Eric Holk</span> how did you make the stderr file? Normally, I just write the ui test file and then run the tests with <code>--bless</code> and have it generate the correct stderr file. If you've pushed your changes to a fork, I can also take a look! <span aria-label="smile" class="emoji emoji-1f642" role="img" title="smile">:smile:</span></p>



<a name="245200480"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/245200480" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Taylor Yu [they/she] <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#245200480">(Jul 07 2021 at 15:40)</a>:</h4>
<p>it's trickier if you have an expected output that you don't yet have code to produce. the test runner uses a bunch of options to erase some line numbers, etc. to make comparisons less fragile</p>



<a name="245214374"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/245214374" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eholk <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#245214374">(Jul 07 2021 at 17:24)</a>:</h4>
<p>I started with using <code>--bless</code>, which solved the problem for the human-readable output, but then the JSON output became an issue. This is for <a href="https://github.com/rust-lang/rust/pull/86639">https://github.com/rust-lang/rust/pull/86639</a>. <span class="user-mention" data-user-id="123856">@Vadim Petrochenkov</span> suggested using <code>// error-pattern</code> which seems to work.</p>



<a name="245217741"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/245217741" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#245217741">(Jul 07 2021 at 17:51)</a>:</h4>
<p>these _unexpected errors_ are IIRC when the ui test is lacking //~| error annotations (which is understandable for command line arguments like in your case, and not errors coming from source code) that are present in the blessed output, it's not just comparing the .stderrs.</p>



<a name="245218108"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/245218108" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#245218108">(Jul 07 2021 at 17:54)</a>:</h4>
<p>btw long time no see eholk <span aria-label="wave" class="emoji emoji-1f44b" role="img" title="wave">:wave:</span> we worked together on brson's mir2wasm compiler 5y ago IIRC</p>



<a name="245218204"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/Tests%20for%20command%20line%20parsing/near/245218204" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eholk <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/Tests.20for.20command.20line.20parsing.html#245218204">(Jul 07 2021 at 17:55)</a>:</h4>
<p>@lqd thanks! I can't believe that was 5 years ago already! I'm excited to be back working on Rust.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>